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2013  WORKPLACE  AND  EQUAL  OPPORTUNITY  SURVEY  OF 

ACTIVE  DUTY  MEMBERS: 

NONRESPONSE  BIAS  ANALYSIS  REPORT 

Executive  Summary 

The  Defense  Manpower  Data  Center  (DMDC)  conducted  several  studies  to  assess  the 
presence  of  nonresponse  bias  (NRB)  in  estimates  from  the  2013  Workplace  and  Equal 
Opportunity  Sur\>ey  of  Active  Duty  Members  ( 2013  WEOA). 

The  objective  of  this  research  was  to  assess  the  extent  of  nonresponse  bias  for  the 
estimated  rate  of  Racial/Ethnic  Harassment/Discrimination  in  the  active  duty  military.  The  level 
of  nonresponse  bias  can  vary  for  every  question  on  the  survey,  but  DMDC  focused  on  the 
Racial/Ethnic  Harassment/Discrimination  rate  because  this  is  the  central  question  on  the  survey. 
Nonresponse  bias  occurs  when  survey  respondents  are  systematically  different  from  the 
nonrespondents.  Nonresponse  bias  can  occur  with  high  or  low  survey  response  rates,  but  the 
decrease  in  survey  response  rates  in  the  past  decade  has  resulted  in  a  greater  focus  on  potential 
NRB.  DMDC  investigated  the  presence  of  nonresponse  bias  using  several  different  methods, 
and  this  paper  summarizes  the  following  methods  and  results: 

1.  Analyze  response  rates  from  2013  WEOA  and  other  related  DMDC  surveys, 

2.  Evaluate  composition  of  sample  compared  with  survey  respondents, 

3.  Use  late  respondents  as  a  proxy  for  nonrespondents, 

4.  Analyze  item  missing  data  for  Racial/Ethnic  Harassment/Discrimination  questions, 

5.  Analyze  whether  past  Racial/Ethnic  Harassment/Discrimination  victims  respond  to 
later  WEOA  surveys  at  different  rates. 

6.  Analyze  mean  Armed  Forces  Qualification  Test  scores  between  active  duty 
population  and  2013  WEOA  survey  respondents. 


The  six  studies  provide  little  evidence  of  nonresponse  bias  in  estimates  of  the 
Racial/Ethnic  Harassment/Discrimination  rate  from  the  2013  WEOA.  The  largest  evidence  of 
nonresponse  bias  is  where  study  five  shows  that  respondents  to  the  prior  WEOA  respond  to  the 
current  WEOA  at  very  high  rates,  regardless  of  their  demographic  characteristics.  If  these 
cooperative  respondents  have  different  attitudes  and  opinions  than  nonrespondents,  this  provides 
limited  evidence  of  nonresponse  bias. 
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2013  WORKPLACE  AND  EQUAL  OPPORTUNITY  SURVEY 
OF  ACTIVE  DUTY  MEMBERS: 
NONRESPONSE  BIAS  ANALYSIS  REPORT 

Introduction  and  Outline 

The  Defense  Manpower  Data  Center  (DMDC)  conducted  several  studies  to  assess  the 
presence  of  nonresponse  bias  in  estimates  from  the  2013  Workplace  and  Equal  Opportunity 
Survey  of  Active  Duty  Members  ( 2013  WEOA). 

The  objective  of  this  research  was  to  assess  the  extent  of  nonresponse  bias  (NRB)  for  the 
estimated  rate  of  Racial/Ethnic  Harassment/Discrimination1  (henceforth  this  rate  will  be  referred 
to  as  Racial  Discrimination)  in  the  active  duty  military.  The  purpose  of  the  Racial 
Discrimination  rate  was  to  provide  the  policy  offices  and  the  Department  with  an  overall 
estimate  of  active  duty  members  who  experienced  behaviors  aligned  with  racial/ethnic 
harassment  and/or  discrimination.  The  level  of  nonresponse  bias  can  vary  for  every  question  on 
the  survey,  but  DMDC  focused  on  the  Racial  Discrimination  rate  because  this  is  the  central 
question  on  the  survey.  Nonresponse  bias  occurs  when  survey  respondents  are  systematically 
different  from  the  nonrespondents.  Statistically,  the  bias  in  a  respondent  mean  (e.g.,  Racial 
Discrimination  rate)  is  a  function  of  the  response  rate  and  the  relationship  (covariance)  between 
response  propensities  and  the  estimated  statistics  (i.e.,  Racial  Discrimination  rate),  and  takes  the 
following  form: 

Bias  (yr)  =  ^  oyop,  where  ayp  =  covariance  between  y  and  response 

propensity,  p 

Nonresponse  bias  can  occur  with  high  or  low  survey  response  rates,  but  the  decrease  in 
survey  response  rates  in  the  past  decade  has  resulted  in  a  greater  focus  on  potential  NRB. 

DMDC  investigated  the  presence  of  nonresponse  bias  using  many  different  methods,  and  this 
paper  summarizes  the  following  methods  and  results: 

1.  Analyze  response  rates  from  2013  WEOA  and  other  related  DMDC  surveys, 

2.  Evaluate  composition  of  sample  compared  with  survey  respondents, 

3.  Use  late  respondents  as  a  proxy  for  nonrespondent, 

4.  Analyze  item  missing  data  for  Racial  Discrimination  questions, 

5.  Analyze  whether  past  Racial  Discrimination  victims  respond  to  later  WEOA  surveys 
at  different  rates. 


’See  Appendix  A  for  the  relevant  survey  questions  and  the  creation  of  this  rate. 
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6.  Analyze  mean  Armed  Forces  Qualification  Test  scores  between  active  duty 
population  and  WEOA  survey  respondents. 

The  first  section  of  this  paper  is  a  summary  of  DMDC’s  nonresponse  bias  results.  The 
second  section  describes  the  2013  WEOA  survey.  The  third  section  consists  of  the  six  individual 
nonresponse  bias  studies.  The  final  section  contains  additional  appendix  figures  including  how 
the  Racial  Discrimination  rate  was  created. 

Summary  of  Findings 

Nonresponse  bias  (NRB)  is  difficult  to  assess.  Most  authors  recommend  averaging 
across  several  different  studies  to  measure  NRB  (Montaquila  and  Olson,  2012).  DMDC  has 
taken  that  approach  here  and  conducted  six  studies  to  assess  NRB  in  Racial  Discrimination 
estimates.  Our  analyses  indicate  that  the  level  of  NRB  in  2013  WEOA  estimates  of  the 
Racial  Discrimination  rate  are  likely  quite  small. 

We  summarize  the  results  from  each  study  below: 

1.  Analyze  response  rates  from  2013  WEOA  and  other  related  DMDC  surveys — 

Analysis  of  response  rates  show  that  comparisons  of  WEOA  and  the  Status  of  Forces 
Survey  of  Active  Duty  Members  (SOFS-A)  provide  potential  evidence  that  topic 
saliency  alters  response  rates  to  the  WEOA  survey,  but  any  increase  in  NRB  over  the 
SOFS-A  is  likely  to  be  small  to  modest. 

2.  Evaluate  composition  of  sample  compared  with  survey  respondents — The  2013 
WEOA  sample  composition  demographically  differs  from  the  active  duty  population 
distribution  due  to  intentional  sampling  strategies  that  allow  DMDC  to  make  precise 
estimates  for  small  subgroups.  The  respondent  composition  differs  from  the  sample 
distribution  in  predictable  ways  due  to  subgroups  (e.g.,  junior  enlisted)  responding  at 
different  rates.  Analyses  show  that  the  survey  weights  effectively  eliminate  these 
differences  and  the  distribution  of  weighted  survey  respondents  closely  matches  the 
active  duty  population. 

3.  Use  late  respondents  as  a  proxy  for  nonrespondents — The  analysis  of  late 
respondents  provides  no  systematic  evidence  of  nonresponse  bias  in  the  estimates  of 
the  Racial  Discrimination  rate.  Late  respondents  are  disproportionately  from  low 
response  rate  groups  and  groups  that  have  higher  Racial  Discrimination  rates,  and 
therefore  we  would  expect  unweighted  rates  to  be  higher  for  late  respondents.  After 
performing  a  weighted  logistic  regression,  the  results  show  that  the  timing  of  the 
returned  survey,  using  late  respondents  as  a  proxy  for  nonrespondents,  is  not 
significant  in  whether  or  not  a  member  experienced  Racial  Discrimination. 

4.  Analyze  item  missing  data  for  Racial  Discrimination  questions — The  questions 
that  contribute  to  the  Racial  Discrimination  rate  showed  no  significant  number  of 
drop-offs  compared  to  other  survey  questions.  The  number  of  drop-offs  for  a 
question  is  driven  more  by  the  length  of  the  question  rather  than  the  sensitive  nature 
of  the  Racial  Discrimination  questions,  an  effect  DMDC  also  observed  when 
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assessing  NRB  in  the  2012  WGRA  survey:  2012  Workplace  and  Gender  Relations 
Surx’ey  of  Active  Duty  Members:  Nonresponse  Bias  Analysis  Report  (DMDC, 

2013d).  The  analysis  of  missing  data  provides  no  evidence  of  nonresponse  bias. 

5.  Analyze  whether  past  Racial  Discrimination  victims  respond  to  later  WEOA 

surveys  at  different  rates — Members  who  reported  experiencing  Racial 
Discrimination  in  an  earlier  survey  appear  equally  likely  to  respond  to  later  WEOA 
surveys.  Additionally,  the  results  of  a  weighted  logistic  regression  show  that  prior 
experience  is  not  significant  in  modelling  response  propensity.  This  study  provides 
no  evidence  of  NRB  in  estimates  of  Racial  Discrimination. 

6.  Analyze  mean  Armed  Forces  Qualification  Test  scores  between  active  duty 
population  and  2013  WEOA  survey  respondents — DMDC  investigated  whether 
respondents  to  the  WEOA  had  systematically  different  AFQT  scores  than 
nonrespondents  after  controlling  (through  weighting)  demographic  differences 
between  survey  respondents  and  nonrespondents.  DMDC  concludes  that  this  study 
provides  very  little  evidence  of  NRB  because  the  weighted  estimates  almost  exactly 
match  the  known  population  values. 

2013  WEOA  Survey 

The  2013  WEOA  survey  sample  size  was  88,816  active  duty  members  selected  from  the 
1,407,767  active  members  on  the  September  2012  Active  Duty  Master  File  (ADMF).  The  frame 
included  Army,  Navy,  Marine  Corps,  Air  Force,  and  Coast  Guard  active  duty  members  who 
were  ranked  El -06  in  September  when  the  survey  fielded.  DMDC  selected  a  stratified  random 
sample  using  the  following  three  characteristics  to  define  the  stratification  dimensions: 
race/ethnicity,2  Service,  and  paygrade.3  Completed  surveys  were  returned  by  18,018  eligible 
sampled  members,  resulting  in  a  23%  weighted  response  rate.  These  respondents  were  weighted 
to  the  full  active  population  using  standard  weighting  methods.  The  four-step  weighting  process 
included: 

1.  Assigning  a  base  weight  based  on  the  inverse  of  the  probability  of  selection, 

2.  Adjusting  the  base  weight  by  distributing  the  weights  from  the  cases  of  unknown 
eligibility  to  the  cases  of  known  eligibility, 

3.  Adjusting  the  weight  from  step  2  by  distributing  the  weights  from  incomplete  cases  to 
the  complete  cases, 

4.  Post-stratifying  the  step  3  weight  to  known  population  totals  for  race/ethnicity, 
Service,  gender,  and  paygrade. 


2  Race/ethnicity  was  stratified  as  a  seven  level  variable:  Hispanic,  White,  Black,  American  Indian/ Alaskan  Native, 
Asian,  Hawaiian/Pacific  Islander,  Multi  Race 

3  Paygrade  was  stratified  as  a  five  level  variable:  E1-E4,  E5-E9,  W1-W5,  01-03,  04-06 
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Applying  the  weights  to  the  respondents,  DMDC  estimated  that  overall  10.2%  (±1.04)  of 
active  duty  military  members  had  experienced  Racial  Discrimination.  DMDC  further  estimated 
that  6.5%  of  non-Hispanic  whites  (±1.5)  and  15.9%  of  minorities  (±1.3)  in  the  active  duty 
military  had  experienced  Racial  Discrimination.  The  statistical  methodology  report  (DMDC, 
2013b)  provides  more  details  regarding  the  sampling,  weighting,  and  variance  estimation  and  the 
tabulation  volume  (DMDC,  2013c)  provides  details  for  the  estimates  of  Racial  Discrimination 
rates  by  additional  demographic  groups. 


4  The  margin  of  error  of  this  estimate  is  based  on  a  95  percent  confidence  interval 
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Section  I: 

Analyze  Response  Rates  From  2013  WEOA  and  Other  Related  DMDC 

Surveys 

DMDC  always  computes  response  rates  by  many  known  administrative  variables  (e.g., 
Service  and  paygrade).  Differential  response  rates  can  be  evidence  of  potential  NRB  unless 
these  variables  are  controlled  for  during  statistical  weighting.  Table  1  shows  that  response  rates 
to  the  WEOA  and  comparable  SOFS-A  vary  greatly  by  subgroup;  for  example,  04-06s 
consistently  respond  at  a  much  higher  rate  than  El-E4s.  Because  04-06s  also  report  very 
different  Racial  Discrimination  rates  than  El-E4s,  NRB  levels  would  be  high  if  DMDC  used 
unweighted  estimates.  However,  DMDC  controls  for  Service,  paygrade,  gender,  race/ethnicity, 
location,  occupation  group,  age  and  on/off  base,  variables  that  are  correlated  with  response 
propensity  as  well  as  actual  survey  responses  when  constructing  survey  weights.  Therefore, 
analysis  of  response  rates  alone  does  not  provide  evidence  of  NRB  in  weighted  2013  WEOA 
estimates.  Instead,  the  focus  of  this  response  rate  analysis  is  to  assess  a  different  hypothesis. 
Some  critics  may  hypothesize  that  minorities,  or  potentially  Racial  Discrimination  victims, 
would  be  more  likely  to  respond  to  the  WEOA  because  of  the  subject  matter,  a  hypothesis 
Groves  (2000)  refers  to  as  topic  saliency.  If  this  were  true,  minorities  should  respond  at  different 
rates  to  the  WEOA  than  they  do  to  other  active  duty  surveys  that  do  not  focus  on  racial  issues. 

To  assess  this  hypothesis,  DMDC  compared  the  2013  WEOA  response  rates  to  the 
previously  fielded  WEOA  survey  and  SOFS-A  that  fielded  in  close  time  proximity.  The  SOFS- 
A  is  DMDC’s  main  recurring  general  topic  survey  that  covers  the  same  active  duty  population  as 
WEOA.  DMDC  used  the  prior  WEOA  survey  ( 2009  WEOA)  and  the  SOFS-A  surveys  that 
fielded  the  closest  to  the  WEOA  surveys,  which  were  in  2012  and  2008.  Table  1  shows  overall 
response  rates  (labeled  “Total”)  and  response  rates  for  key  demographic  subgroups. 

Table  1  shows  that  response  rates  to  the  WEOA  follow  patterns  consistent  with  known 
trends  in  the  SOFS-A.  Over  time,  across  all  military  surveys,  active  duty  response  rates  have 
steadily  declined.  The  WEOA  shows  a  more  severe  decline  than  the  SOFS-A;  however,  this  can 
be  attributed  to  budget  pressures  that  forced  the  removal  of  the  WEOA  paper  survey  option  after 
the  2009  cycle.5 


5  The  2009  WEOA  surveys  had  paper  and  Web  response  options  while  the  2013  survey  was  Web-only. 
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Table  1. 

Comparison  of  Trends  in  WEOA  and  SOFS-A  Response  Rates  ( Shown  in  Percent ) 


2008/2009 

2012/2013 

Key  Surveys 

WEOAa 

KIM: 

WEOA 

SOFS-Ac 

Total 

33 

31 

23 

26 

Race/Ethnicity 

Non-Hispanic  White 

35 

33 

24 

28 

Minority 

30 

28 

22 

22 

Black 

29 

27 

21 

21 

Hispanic 

28 

28 

21 

22 

Asian 

36 

32 

26 

28 

All  Other  Races 

29 

24 

21 

22 

Service 

Army 

29 

26 

17 

20 

Navy 

33 

32 

25 

27 

Marine  Corps 

21 

23 

20 

21 

Air  Force 

42 

42 

32 

37 

Coast  Guard 

55 

44 

48 

Paygrade 

E1-E4 

17 

16 

12 

13 

E5-E9 

41 

39 

29 

33 

01-03 

43 

42 

33 

38 

04-06 

60 

60 

45 

54 

Gender 

Male 

33 

31 

23 

26 

Female 

35 

34 

26 

29 

aThe  2009  WEOA  surveys  had  paper  and  Web  response  options  while  the  2013  survey  was  Web-only. 

bThe  November  2008  SOFS-A  was  used  because  it  was  the  most  recent  SOFS-A  survey  prior  to  the  2009  WEOA,  which  was  conducted  in 
February  2009 

cThe  June  2012  SOFS-A  was  used  because  it  was  the  most  recent  SOFS-A  survey  prior  to  the  2013  WEOA,  which  was  conducted  in  April  2013 


For  race/ethnicity,  non-Hispanic  whites  and  Asians  consistently  respond  to  active  duty 
surveys  at  higher  rates  than  other  minorities.  However,  comparing  the  most  recent  WEOA  and 
SOFS-A  surveys  reveals  that  the  response  rate  gap  between  non-Hispanic  whites  and  minorities 
is  smaller  in  the  2013  WEOA  survey.  For  example,  response  rates  for  non-Hispanic  whites 
(28%)  were  six  percentage  points  higher  than  minorities  (22%)  in  the  June  2012  SOFS-A,  but 
response  rates  for  non-Hispanic  whites  were  only  two  percentage  points  higher  in  the  2013 
WEOA  (24%  versus  22%).  This  may  indicate  that  the  subject  matter  of  equal  opportunity 
influences  some  minorities  to  respond  (topic  saliency)  or  may  dissuade  some  non-Hispanic 
whites  from  responding  because  of  lack  of  topic  interest.  The  decrease  in  the  race/ethnicity  gap 
is  a  consideration  but  does  not  necessarily  indicate  an  increase  in  NRB  because  race/ethnicity  is 
a  characteristic  that  is  controlled  for  during  survey  weighting.  Therefore  the  only  way  that  the 
smaller  race/ethnicity  gap  could  create  larger  NRB  is  if  the  minorities  that  were  influenced  to 
respond  had  higher  (or  lower)  Racial  Discrimination  rates  than  those  that  did  not  respond,  and 
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that  hypothesis  is  not  testable  with  these  data.  However,  the  presence  of  this  gap  could  lead  to 
slightly  increased  risk  for  NRB  in  WEOA  surveys. 

For  Service,  response  rate  patterns  are  consistent  between  the  SOFS-A  and  WEOA 
surveys  across  years.  Air  Force  response  rates  are  highest,  followed  by  Navy,  and  the  lowest 
response  rates  belong  to  Army  and  Marine  Corps.  The  response  rates  by  Service  provide  no 
evidence  of  additional  NRB  in  the  WEOA  survey  that  does  not  exist  in  the  SOFS-A. 

For  paygrade,  response  rate  patterns  are  consistent  across  all  surveys  where  senior 
officers  (04-06)  respond  at  the  highest  rates  and  response  rates  decrease  as  active  members 
become  more  junior  until  dropping  off  rapidly  for  the  junior  enlisted  (E1-E4).  DMDC’s 
weighting  methods  correct  for  bias  associated  with  the  differential  response  probabilities  for 
these  known  characteristics  (e.g.,  Service,  paygrade).  The  response  rates  by  paygrade  provide  no 
evidence  of  additional  NRB  in  the  WEOA  survey  that  does  not  exist  in  the  SOFS-A. 

Summary  of  Response  Rates  Analysis  From  2013  WEOA  and  Other  Related 

DMDC  Surveys 

Comparisons  of  WEOA  and  SOFS-A  response  rates  provide  evidence  that  topic  saliency 
does  not  substantially  alter  response  rates  to  the  WEOA  survey,  and  therefore  any  increase  in 
NRB,  compared  to  that  of  a  SOFS-A,  is  likely  to  be  small  to  modest.  However,  although  WEOA 
and  SOFS-A  response  rates  have  similar  patterns,  the  difference  between  non-Hispanic  white 
and  minority  response  rates  (race/ethnicity  gap)  suggests  that  topic  saliency  may  increase  the 
level  of  NRB  in  the  WEOA  over  the  SOFS-A,  but  because  the  response  rate  gap  is  only  slightly 
smaller  for  WEOA,  the  increase  in  NRB  is  likely  small. 
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Section  II: 

Evaluate  Composition  of  Sample 
Compared  With  Survey  Respondents 

DMDC  next  considered  whether,  and  how,  survey  nonresponse  (unit  nonresponse), 
affects  NRB  for  this  survey.  In  this  section  DMDC  evaluates  the  composition  of  the  2013 
WEOA,  exploring  key  military  demographic  breakdowns  by  survey  subgroups  (e.g.,  population 
total,  sample  size,  respondents,  and  weighted  respondents).  DMDC  draws  optimized  samples  to 
reduce  survey  burden  on  members  as  well  as  produce  high  levels  of  precision  for  important 
domain  estimates  by  using  known  information  about  the  military  population  and  their  propensity 
to  respond.  It  is  important  to  note  that  DMDC  samples  are  often  not  proportional  to  their 
respective  population.  Depending  on  the  specific  subgroup,  DMDC  will  over  or  under  sample  a 
specific  group  (e.g.,  E1-E4  Army)  to  obtain  enough  expected  responses  to  make  statistically 
accurate  estimates.  While  the  sample  and  the  number  of  responses  might  look  out  of  alignment 
with  the  population,  this  is  by  design.  DMDC  is  able  to  use  its  military  personnel  data  to 
correctly  weight  the  respondents  in  order  to  make  survey  estimates  representative  of  the  entire 
active  duty  population.  The  military  demographics  considered  include:  Service,  paygrade, 
gender,  and  race/ethnicity.  Table  2  through  Table  5  contains  both  the  frequency  and  percent  for 
the  survey  population,  sample  size,  and  respondents  (unweighted  and  weighted)  by  demographic 
category. 

Table  2  shows  the  breakdown  by  race/ethnicity.  Minority  members  typically  have  lower 
response  rates  because  they  are  composed  of  more  junior  enlisted.  For  the  2013  WEOA , 
minorities  were  significantly  oversampled  considering  they  are  disproportionately  victims  of 
Racial  Discrimination.  Overall  minorities  made  up  86%  of  the  sample  compared  to  35%  of  the 
overall  active  duty  military  population.  The  final  weighted  population  pulls  the  respondents 
back  into  alignment  with  race/ethnicity  composition  in  the  active  duty  to  ensure  final  weighted 
estimates  are  not  over-representing  minorities. 


Table  2. 

Distribution  of  Population,  Sample  and  Respondents,  by  Race/Ethnicity 


Population 

Sample 

Respondents 

Weighted  Population 

Race/Ethnicity 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Non-Hispanic 

White 

908,984 

65 

12,718 

14 

3,153 

18 

908,984 

65 

Minority 

498,783 

35 

76,098 

86 

14,865 

83 

498,783 

35 

Black 

223,208 

16 

16,151 

18 

2,961 

16 

223,208 

16 

Hispanic 

159,842 

11 

14,825 

17 

2,840 

16 

159,842 

11 

Asian 

49,489 

4 

11,421 

13 

2,632 

15 

49,489 

4 

All  Other 
Races 

66,244 

5 

33,701 

38 

6,432 

36 

66,244 

5 

Total 

1,407,767 

100 

88,816 

100 

18,018 

100 

1,407,767 

100 
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Table  3  shows  the  breakdown  by  Service.  Based  on  historically  different  response  rates 
and  the  need  to  make  estimates  for  each  Service,  DMDC  oversampled  the  Navy,  Marine  Coips, 
and  Coast  Guard,  and  under  sampled  the  Air  Force  and  Army  (Table  3:  columns  3  and  5).  For 
instance,  Army  makes  up  38%  of  the  active  duty  but  only  33%  of  the  2013  WEOA  sample. 

There  are  fairly  large  differences  between  the  sample  size  and  respondents  percentages, 
especially  with  the  Air  Force  and  Army  (Table  3:  columns  5  and  7).  The  Air  Force  is  the  highest 
responding  group  and  made  up  14%  of  the  sample,  but  21%  of  the  respondents.  Army,  on  the 
other  hand,  made  up  33%  of  the  sample  and  only  24%  of  the  respondents.  Finally,  DMDC  uses 
post-survey  weighting  procedures  (described  earlier)  to  adjust  the  24%  of  Army  respondents  to 
make  them  representative  of  the  Army’s  38%  of  the  overall  military  population.  The  final 
weighting  procedure  aligns  respondent  proportions  back  with  the  military  population  for  Service 
(Table  3:  columns  3  and  9). 6 


Table  3. 

Distribution  of  Population,  Sample  and  Respondents,  by  Service 


Population 

Sample 

Respondents 

Weighted  Population 

Service 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Army 

537,215 

38 

29,581 

33 

4,357 

24 

537,215 

38 

Navy 

310,406 

22 

25,448 

29 

4,686 

26 

310,406 

22 

Air  Force 

324,789 

23 

12,212 

14 

3,793 

21 

324,789 

23 

Marine  Corps 

193,891 

14 

16,793 

19 

3,139 

17 

193,891 

14 

Coast  Guard 

41,466 

3 

4,782 

5 

2,043 

11 

41,466 

3 

Total 

1,407,767 

100 

88,816 

100 

18,018 

100 

1,407,767 

100 

Table  4  shows  the  breakdown  by  paygrade.  Junior  enlisted  members  (E1-E4)  are  known 
to  have  the  lowest  response  rates  for  all  military  surveys.  DMDC  oversamples  this  group  to 
provide  enough  responses  to  make  precise  estimates  for  this  subgroup  (56%  of  the  sample  versus 
44%  of  the  population).  The  lower  response  rate  for  the  E1-E4  group  is  shown  by  them  making 
up  only  33%  of  the  total  respondents.  Higher  responding  groups  such  as  high  ranking  officers 
(04-06)  or  senior  enlisted  members  (E5-E9)  are  under  sampled.  The  high  response  rates  among 
these  specific  subgroups  provide  a  sufficient  number  of  respondents.  The  respondents  DMDC 
received  for  the  2013  WEOA  are  consistent  with  expected  rates  based  on  historical  trends. 

Again,  the  post-stratification  adjustment  properly  aligns  the  final  weighted  population  (Table  4: 
column  9)  with  the  population  (Table  4:  column  3). 


6During  the  2013  WEOA ,  DMDC  controlled  for  race,  Service,  gender,  and  paygrade  during  the  post-stratification 
weighting  stage. 
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Table  4. 

Distribution  of  Population,  Sample  and  Respondents,  by  Pay  grade 


Population 

Sample 

Respondents 

Weighted  Population 

Paygrade 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

E1-E4 

613,841 

44 

50,029 

56 

5,910 

33 

613,893 

44 

E5-E9 

549,786 

39 

28,606 

32 

8,221 

46 

547,720 

39 

W1-W5 

21,116 

2 

1,016 

1 

450 

3 

21,211 

2 

01-03 

134,009 

10 

6,131 

7 

2,059 

11 

135,640 

10 

04-06 

89,015 

6 

3,034 

3 

1,378 

8 

89,303 

6 

Total 

1,407,767 

100 

88,816 

100 

18,018 

100 

1,407,767 

100 

Table  5  shows  the  survey  subgroup  breakdown  by  gender.  The  respondents  DMDC 
received  for  the  2013  WEOA  are  consistent  with  expected  rates  based  on  historical  trends. 
Females  responded  to  the  2013  WEOA  at  slightly  higher  rates  (19%  of  respondents  versus  18% 
of  sample),  but  in  general  Table  5  shows  that  an  assessment  of  gender  shows  no  evidence  of 
NRB. 


Table  5. 

Distribution  of  Population,  Sample  and  Respondents,  by  Gender 


Population 

Sample 

Respondents 

Weighted  Population 

Gender 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Male 

1,202,059 

85 

73,159 

82 

14,570 

81 

1,202,088 

85 

Female 

205,708 

15 

15,657 

18 

3,448 

19 

205,679 

15 

Total 

1,407,767 

100 

88,816 

100 

18,018 

100 

1,407,767 

100 

Summary  of  Sample  Composition  Compared  With  Survey  Respondents 

The  WEOA  sample  composition  demographic  ally  differs  from  the  active  duty  population 
distribution  due  to  intentional  sampling  strategies  that  allow  precise  estimation  for  small 
subgroups  (e.g.,  racial/ethnic  groups).  The  respondent  composition  differs  from  the  sample 
distribution  in  predictable  ways  due  to  subgroups  (e.g.,  junior  enlisted)  responding  at  different 
rates.  Analyses  show  that  the  survey  weights  effectively  eliminate  these  differences  and  the 
distribution  of  weighted  survey  respondents  closely  matches  the  active  duty  population.  The 
difference  in  the  composition  of  the  respondents  compared  with  the  population  distributions  is 
effectively  eliminated  during  survey  weighting.  This  assessment  shows  a  risk  of  NRB  due  to 
differential  response  rates,  but  because  the  differences  are  on  observable  characteristics  (e.g., 
Service,  paygrade)  the  weighting  eliminates  NRB  for  these  estimates,  and  reduces  NRB  for 
statistics  (e.g.,  Racial  Discrimination)  correlated  with  these  demographics.  DMDC  concluded 
that  although  large  differential  response  rates  provide  great  risk  of  NRB,  the  abundant  frame  data 
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on  military  personnel  allows  complex  weighting  adjustments  to  account  for  a  large  number  of 
observable  characteristics,  and  therefore  this  study  provides  no  evidence  of  NRB  in  the  2013 
WEOA  estimates. 
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Section  III: 

Use  Late  Respondents  as  a  Proxy  for  Nonrespondents 


Survey  researchers  have  observed  that  if  the  field  period  were  shortened  or  fewer  contact 
attempts  were  used,  a  subset  of  survey  respondents  would  have  been  nonrespondents,  and  they 
have  hypothesized  that  these  late  respondents  may  be  more  similar  to  nonrespondents  than  the 
early  respondents.  This  hypothesis  is  called  the  “continuum  of  resistance”  model  (Lin  & 
Schaeffer,  1995).  Although  results  from  studies  testing  this  model  have  been  mixed  (Groves  & 
Peytcheva,  2008),  analysis  of  late  respondents  is  still  a  common  practice  in  NRB  studies. 

DMDC  evaluated  the  effect  of  late  responders  by  performing  a  weighted  logistic 
regression  using  PROC  SURVEYLOGISTIC  in  SAS.  Specifically,  DMDC  assessed  whether  a 
dichotomous  predictor  variable  for  early/late  response  was  a  significant  predictor  of  Racial 
Discrimination,  after  controlling  for  other  co variates.  If  late  respondents  report  different 
experiences  from  early  respondents,  the  early/late  predictor  variable  should  be  significant,  and 
may  provide  evidence  of  NRB  if  late  responders  are  similar  to  survey  nonrespondents.  Note  that 
whether  late  respondents  are  similar  to  nonrespondents  on  estimates  of  interest  cannot  be  directly 
measured,  but  whether  they  are  similar  on  observable  characteristics  using  administrative 
variables  can  be  assessed. 

Table  6  shows  the  number  of  respondents  by  week  of  fielding.  To  define  early  and  late 
respondents,  DMDC  divided  the  fifteen  week  field  period  into  two  parts,  treating  respondents 

7 

from  the  first  twelve  weeks  as  early  respondents  and  the  final  three  weeks  as  late  respondents. 


7  The  choice  for  breaking  the  field  period  into  early  and  late  respondents  is  subjective.  We  chose  the  final  two 
weeks  to  coincide  with  the  final  survey  contact  and  to  ensure  there  were  sufficient  numbers  of  late  respondents  to 
make  separate  estimates  with  reasonable  precision. 
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Table  6. 

Respondents  by  Week  of  Fielding 


Early/Late  Split 

Week 

Respondents 

Early 

1 

4,806 

2 

3,478 

3 

1,667 

4 

1,361 

5 

741 

6 

1,009 

7 

518 

8 

295 

9 

1,035 

10 

432 

11 

763 

12 

187 

Late 

13 

948 

14 

763 

15 

15 

Total 

18,018 

Table  7  shows  the  demographic  composition  for  early  respondents,  late  respondents,  and 
nonrespondents  by  race/ethnicity,  Service,  and  paygrade,  and  gender. 
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Table  7. 

Composition  of  Sample  for  Early,  Late,  and  Nonrespondents 


Key 

Domains 

Early  Respondents 

Late  Respondents 

Nonrespondents 

Number  of 
Respondents 

Unweighted 
Percent  of 
Total  Early 
Respondents2 

Number  of 
Respondents 

Unweighted 
Percent  of 
Total  Late 
Respondents2 

Number  of 
Nonrespondents 

Unweighted 
Percent  of  Total 
Nonrespondents2 

Race/Ethnicity 

Non- 

Hispanic 

Whites 

2,885 

18 

268 

16 

9,087 

14 

Minority 

13,407 

82 

1,458 

84 

58,123 

86 

Black 

2,628 

16 

333 

19 

12,439 

19 

Hispanic 

2,569 

16 

271 

16 

11,437 

17 

Asian 

2,382 

15 

250 

14 

8,295 

12 

All  Other 
Races 

5,828 

36 

604 

35 

25,952 

39 

Service 

Army 

3,828 

24 

529 

31 

24,114 

36 

Navy 

4,259 

26 

427 

25 

19,812 

29 

Marine 

Corps 

2,865 

18 

274 

16 

12,956 

19 

Air  Force 

3,440 

21 

353 

20 

7,831 

12 

Coast 

Guard 

1,900 

12 

143 

8 

2,497 

4 

Paygrade 

E1-E4 

5,247 

32 

663 

38 

42,264 

63 

E5-E9 

7,459 

46 

762 

44 

19,068 

28 

W1-W5 

415 

3 

35 

2 

530 

1 

01-03 

1,887 

12 

172 

10 

3,826 

6 

04-06 

1,284 

8 

94 

5 

1,522 

2 

Gender 

Male 

13,177 

81 

1,393 

81 

55,827 

83 

Female 

3,115 

19 

333 

19 

11,383 

17 

Total 

16,292 

100 

1,726 

100 

67,210 

100 

aDetails  may  not  add  to  totals  because  of  rounding. 


Early  and  late  respondents  generally  look  demographically  similar;  however,  late 
respondents  contain  a  lower  percentage  of  Coast  Guard  (8%  versus  12%),  more  Army  (31% 
versus  24%),  and  more  E1-E4  (38%  versus  32%).  2013  WEOA  late  respondents  are  more 
demographically  similar  to  the  nonrespondents  than  the  early  respondents,  but  they  are  still 
demographically  different  from  the  nonrespondents.  For  instance,  late  respondents  are 
disproportionately  E1-E4  relative  to  early  respondents,  but  nonrespondents  are  much  more  El- 
E4  than  late  respondents  (63%  compared  with  38%).  The  pattern  follows  for  Service  and 
race/ethnicity,  where  late  respondents  are  more  Army  and  minority,  and  then  the  effect  is  more 
pronounced  for  nonrespondents  (e.g.,  36%  Army  for  nonrespondents  versus  31%  for  late 
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respondents,  86%  minority  for  nonrespondents  versus  84%  for  late  respondents).  For  gender, 
nonrespondents  look  very  similar  to  both  early  and  late  respondents.  While  the  analysis  of  the 
demographics  shows  that  late  respondents  do  look  “more  like”  nonrespondents,  which  provides 
limited  support  for  the  continuum  of  resistance  model,  early,  late,  and  nonrespondents  are  still 
quite  different  from  one  another.  Next,  we  investigate  Racial  Discrimination  propensity  through 
logistic  regression  using  key  predictor  variables  including  the  early/late  response  variable. 

Respondents  and  nonrespondents  are  characterized  based  on  a  set  of  demographic 
variables.  Variables  such  as  member’s  race/ethnicity,  gender,  paygrade,  and  Service  can  be 
critical  in  predicting  military  experience  of  Racial  Discrimination.  The  analysis  of  Racial 
Discrimination  was  conducted  via  logistic  regression  with  the  nine  independent  variables  shown 
in  Table  8.  The  dependent  variable  of  the  logistic  model  is  a  binary  variable  representing 
whether  or  not  the  member  experienced  Racial  Discrimination  where  the  variable  equals  0  for  no 
experience  and  1  for  experience.  Although  variables  such  as  Service  and  paygrade  are  important 
predictors,  early/late  response  is  the  variable  of  interest.  Most  of  the  variables  in  the  table  are 
self  explanatory  with  the  exception  of  occupation  group.  The  groups  for  occupation  were 
determined  based  on  historical  response  rates,  where  DMDC  coded  specific  occupation  groups 
as  low,  average,  and  high  response  rate  groups. 
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Table  8. 

Independent  Demographic  Variables  for  Logistic  Model  Predicting  Racial  Discrimination 


Variable 

Categories 

Early/Late 

Late  Responder 

Race/Ethnicity 

Hispanic 

Black 

Asian 

All  Other  Races 

Gender 

Male* 

Female 

Paygrade 

E1-E4* 

E5-E9 

W1-W5 

01-03 

04-06 

Service 

Army* 

Navy 

Marine  Corps 

Air  Force 

Coast  Guard 

Location 

US  &  US  territories* 

Asia  &  Pacific  Islands 

Europe 

Age 

Under  25  Years  Old 

25-29  Years  Old 

30-35  Years  Old 

36-44  Years  Old 

45+  Years  Old 

Occupation  Group 

Low  Response  Rate 
Occupations* 

Average  Response  Rate 
Occupations 

High  Response  Rate 
Occupations 

On/Off  base 

On  Base* 

Off  Base 

Represents  the  reference  category  for  each  variable. 


The  purpose  of  testing  the  full  model  was  to  measure  the  effect  of  each  variable  on  Racial 
Discrimination  while  controlling  for  the  others  (i.e.  measuring  the  effect  of  one  characteristic 
taking  the  other  characteristics  into  consideration).  To  perform  statistical  modeling  using 
logistic  regression,  we  set  one  of  the  categories  (levels)  of  the  independent  variable  to  be  a 
reference  category,  shown  with  an  asterisk  (  )  in  Table  8.  DMDC  modeled  the  data  using  SAS 
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PROC  SURVEYLOGISTIC.  All  other  categories  of  the  variable  were  compared  with  the 
reference  category  and  the  model  parameters  and  odds  ratios  were  derived  and  interpreted 
accordingly.  The  odds  ratio  can  be  interpreted  as  the  odds  that  an  outcome  (in  this  case 
experiencing  Racial  Discrimination)  will  occur  given  a  non-reference  category  compared  to  the 
odds  of  that  outcome  for  the  reference  category.. 

Table  9  displays  the  output  statistics  from  the  weighted  full  logistic  model.  Column  1 
shows  the  independent  variables  and  their  categories.  The  second  through  fifth  columns  consist 
of  the  parameter  estimates,  the  standard  errors  of  the  estimate,  the  Wald  tests,  and  the  degrees  of 
freedom  associated  with  the  variables  and  categories,  respectively.  Wald’s  test  and  the 
corresponding  p-values  for  Air  Force,  Hispanic,  Black,  and  Asian  are  significant,  suggesting  that 
these  levels  of  variables  exhibit  significant  power  for  predicting  Racial  Discrimination 
experience.  Minority  groups  are  expected  to  report  more  Harassment/Discrimination,  and  the 
odds  ratios  show  that  minorities  experience  2  to  3  times  the  rate  of  non-Hispanic  whites 
(reference  group). 
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Table  9. 

Logistic  Model  Predicting  Racial  Discrimination  with  Nine  Independent  Variables 
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The  odds  ratio  for  each  variable  in  the  model  is  interpreted  taking  the  impact  of  the  other 
variables  in  the  model  into  consideration.  For  example,  the  odds  ratio  for  race/ethnicity  level 
Hispanic  is  2.703,  indicating  that  Hispanic  members  are  about  3  times  as  likely  as  non-Hispanic 
whites  to  experience  Racial  Discrimination  after  controlling  for  the  other  variables  in  the  model. 

Table  9  shows  that  the  early/late  predictor  variable  is  not  significant  (p-value  is  0.8005) 
and  the  odds  ratio  is  only  1.042.  This  shows  that  the  late  responders  experience  Racial 
Discrimination  at  almost  the  exact  same  rate  as  early  responders,  after  controlling  for 

o 

demographic  differences  between  the  two  groups. 

Table  10  shows  the  composition  of  early/late  respondents  broken  down  by  race/ethnicity. 
Additionally,  the  table  shows  the  number  of  unweighted  reports  of  Racial  Discrimination  cases 
and  the  unweighted  rates  by  race/ethnicity.  The  late  respondents  report  higher  overall 
unweighted  Racial  Discrimination  rates  (14.4%  versus  12.2%),  and  higher  for  each  race/ethnicity 
group;  however,  this  is  expected  because  later  respondents  are  disproportionately  high  risk 
groups  (e.g.,  E1-E4). 


Table  10. 

Comparison  of  Early  and  Late  Respondents  by  Race/Ethnicity  for  Racial  Discrimination 
Cases  and  Unweighted  Rates 


Time  Period 

Race 

Respondents 

Unweighted  Racial 
Discrimination  Cases 

Racial  Discrimination 
Unweighted  Rate 
(Percent) 

Early  Respondents 

Non-Hispanic 

White 

2,885 

145 

5.0 

Minority 

13,407 

1,839 

13.7 

Black 

2,628 

426 

16.2 

Hispanic 

2,569 

388 

15.1 

Asian 

2,382 

409 

17.2 

All  Other  Races 

5,828 

616 

10.6 

Total 

16,292 

1,984 

12.2 

Late  Respondents 

Non-Hispanic 

White 

268 

16 

6.0 

Minority 

1,458 

233 

16.0 

Black 

333 

62 

18.6 

Hispanic 

271 

47 

17.3 

Asian 

250 

47 

18.8 

All  Other  Races 

604 

77 

12.7 

Total 

1,726 

249 

14.4 

8  An  unweighted  model  was  also  ran  to  test  the  sensitivity  of  the  weights  on  the  estimated  parameters,  but  the 
early/late  predictor  variable  was  still  not  significant,  and  the  odds  ratio  was  only  slightly  higher  at  1.126. 
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Summary  of  Using  Late  Respondents  as  a  Proxy  for  Nonrespondents 


Observing  the  unweighted  Racial  Discrimination  rates  in  Table  10,  the  late  respondents 
have  higher  rates  (14.4%  versus  12.2%)  than  early  respondents.  Because  there  is  little  difference 
in  non-Hispanic  whites  (5.0%  for  early  and  6.0%  for  late  respondents)  and  each  level  of 
minorities  presenting  higher  rates,  there  may  be  some  concern  for  NRB.  However,  due  to  late 
respondents  being  composed  primarily  of  low  response  rate  groups,  as  seen  in  Table  7  (e.g.,  El- 
E4,  minorities),  who  also  have  higher  Racial  Discrimination  rates,  this  increase  is  expected. 

Additionally,  the  analysis  of  late  respondents  using  the  logistic  regression  model  provides 
no  significant  evidence  of  NRB  in  the  estimates  of  the  Racial  Discrimination  rate.  The  model 
controlled  for  the  demographic  differences,  and  the  early/late  predictor  variable  was  not 
significant  in  predicting  whether  a  sample  member  experienced  Racial  Discrimination. 

Therefore,  if  late  respondents  serve  as  proxies  for  survey  nonrespondents,  then  there  is  no 
evidence  that  nonrespondents  would  have  different  Racial  Discrimination  rates. 


21 


Section  IV: 

Analyze  Item  Missing  Data  for  Racial  Discrimination  Questions 

In  this  section,  we  analyze  item  missing  data  for  the  Racial  Discrimination  questions  to 
investigate  the  hypothesis  that  some  respondents  refuse  to  answer  questions  or  quit  the  survey  all 
together  (i.e.,  drop-off)  because  of  the  sensitivity  of  the  questions.  If  the  decision  to  refuse  to 
answer  the  question  is  not  random  (i.e.,  those  who  avoid  the  Racial  Discrimination  questions 
have  different  harassment  rates  than  complete  respondents),  then  a  source  of  NRB  exists.  We 
cannot  directly  test  this  hypothesis  because  the  Racial  Discrimination  status  for  respondents  that 
avoid  the  question  is  unknown.  However,  we  examine  item  missing  data  to  assess  the  NRB  in 
the  Racial  Discrimination  questions. 

To  understand  whether  respondents  specifically  avoided  the  Racial  Discrimination 
questions,  or  whether  they  quit  the  survey  prior  to  ever  seeing  the  questions,  DMDC  conducted  a 
“drop-off  analysis”.  Our  drop-off  analysis  shows  the  last  question  that  a  survey  respondent 
answered  on  the  survey.  For  example,  if  a  respondent  answered  Ql-10  and  quit,  the  drop-off 
analysis  would  place  that  respondent  in  the  frequency  count  at  Q10.  Drop-off  analysis  does  not 
account  for  “standard  item  missing  data”,  for  instance  when  a  respondent  skips  one  question 
(accidentally  or  on  purpose),  but  returns  to  answer  further  questions.  For  instance,  if  a  member 
answered  Ql-10,  skipped  to  12  and  answered  Q 12-20,  and  then  answered  no  further  questions, 
the  drop-off  analysis  would  include  the  member  in  the  count  where  Q20  was  last  answered. 

In  the  2013  WEOA  survey,  there  were  only  fifteen  questions  on  the  web  survey  where  a 
large  number  of  respondents  (over  100)  dropped  off.  Of  these  fifteen  questions,  four  were 
directly  related  to  the  Racial  Discrimination  rate  (Questions  28,  29,  31,  and  32,  See  Appendix 
A).  However,  this  does  not  prove  that  the  subject  matter  of  equal  opportunity  was  the  cause  for 
the  drop-off.  Another  reason  respondents  may  drop  out  of  the  survey  is  survey  burden,  as 
measured  by  question  length.  Table  1 1  breaks  down  the  fifteen  questions  with  large  drop-offs  by 
showing  the  amount  of  drop-offs  as  well  as  the  number  of  sub  items  for  the  following  question. 
Of  the  fifteen  major  instances,  thirteen  show  that  the  following  question  involved  multiple  sub 
items,  and  these  long  sub  items  may  appeal-  burdensome  to  respondents. 
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Table  11. 

Breakdown  of  Large  Drop-off  Questions 


Last  Question 
Answered 

Number  of  Drop-offs 

Number  of  Sub  items 
in  Next  Question 

8 

250 

1 

10 

165 

6 

13 

284 

12 

18 

211 

9 

25 

192 

10 

26 

190 

8 

27 

*347 

18 

28 

*241 

1 

30 

*113 

21 

31 

*117 

4 

33 

477 

8 

50 

114 

16 

55 

156 

5 

58 

395 

10 

60 

158 

7 

^Indicates  the  number  of  drop-offs  when  arriving  at  the  four  questions  that  lead  into  the  Racial  Discrimination  rate. 


Specifically,  two  of  the  Racial  Discrimination  questions  have  the  most  sub  items  with  18 
and  21  levels,  but  do  not  represent  the  most  drop-offs  within  the  survey.  DMDC  also  observed 
that  large  sets  of  questions  presented  on  a  single  web  screen  induced  drop  offs  during  the  2012 
Workplace  and  Gender  Relations  Survey  of  Active  Duty  Members:  Nonresponse  Bias  Analysis 
Report  NRB  (DMDC  2013d). 

Summary  of  Analyzing  Item  Missing  Data  for  Racial  Discrimination  Question 

Similar  to  all  DMDC  surveys,  unit  missing  data  (members  that  fail  to  start  the  survey)  is 
a  much  more  severe  problem  than  item  missing  data  (skipping  questions  on  the  survey),  but  we 
investigated  the  item  missing  data  for  the  Racial  Discrimination  questions  in  search  of  potential 
NRB.  Although  numerous  members  dropped  off  at  the  key  questions  that  lead  to  the  Racial 
Discrimination  rate,  the  long  series  of  scale  questions  (e.g.,  respondent  burden)  for  both  Q28 
(harassment)  and  Q3 1  (discrimination)  seem  to  cause  the  missing  data. 
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Section  V: 

Analyze  Whether  Past  Racial  Discrimination  Victims  Respond  to  Later 

WEOA  Surveys  at  Different  Rates 

NRB  occurs  when  survey  respondents  would  report  different  experiences  than  survey 
nonrespondents.  DMDC  has  historical  data  to  assess  whether  prior  Racial  Discrimination 
victims9  respond  to  future  WEOA  surveys  at  different  rates  than  non-victims.  For  example,  if 
members  who  reported  experiencing  Racial  Discrimination  on  the  2009  WEOA  responded  to  the 
2013  survey  at  significantly  higher  or  lower  rates  than  members  who  reported  no  Racial 
Discrimination  experience,  this  may  suggest  NRB  exists  in  the  2013  WEOA  Racial 
Discrimination  estimates.  Some  critics  may  argue  that  members  who  have  experienced  this 
situation  in  the  past  are  more  likely  to  respond  to  tell  the  story.  For  the  NRB  to  occur,  the  effect 
of  a  4-year  old  Racial  Discrimination  victimization  on  current  survey  response  (e.g.,  2009 
victimization  affecting  2013  response)  would  need  to  be  similar  to  the  effect  of  a  recent 
victimization  (within  last  12  months)  on  response  propensity  to  the  current  survey.  Note  that  we 
cannot  test  this  assumption  with  the  data. 

For  the  survey  iterations  available  (2009  and  2013),  DMDC  traced  the  distribution  of 
members  by  race/ethnicity,  Service,  paygrade,  and  gender.  DMDC  sampled  87,302  members  in 
the  2009  survey  of  which  26,167  were  complete  respondents.  DMDC  then  sampled  3,757  of  the 
2009  respondents  in  the  2013  survey.  The  3,757  respondents  from  the  earlier  administration  that 
were  sampled  again  in  the  later  administration  are  shown  in  Table  12  broken  down  by  their 
response  to  the  Racial  Discrimination  question  in  the  2009  survey  (experienced  Racial 
Discrimination  or  did  not  experience  Racial  Discrimination).  Table  12  also  displays  the 
unweighted  and  weighted  response  rates  for  each  subgroup.  The  weighted  response  rates  were 
based  on  the  sampling  weights  from  the  2013  WEOA  survey. 

DMDC  also  conducted  this  analysis  for  our  2012  Workplace  and  Gender  Relations 
Surx’ey  (DMDC  Report  No.  2013-059),  and  an  important  conclusion  can  be  drawn  across  both 
studies.  Prior  survey  respondents,  whether  harassed  or  not  (either  due  to  gender  or  race), 
respond  to  future  surveys  at  very  high  rates.  What  this  implies  is  that  even  after  conditioning  on 
Service,  paygrade,  race,  gender,  and  many  other  variables,  there  are  a  subset  of  Service  members 
that  are  extremely  cooperative  (i.e.,  take  surveys),  which  also  means  there  also  exists  a  set  of 
non-cooperative  Service  members.  Because  these  two  subgroups  cannot  be  identified  through 
observable  characteristics,  DMDC  is  unable  to  properly  account  for  them  during  weighting. 
Therefore,  if  these  cooperative  members  have  attitudes/opinions  that  differ  from  the 
uncooperative,  this  analysis  provides  evidence  of  NRB. 


9  Prior  Racial  Discrimination  victims  reported  a  Racial  Discrimination  experience  on  a  previous  administration  of 
the  survey. 
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Table  12. 

Demographic  Breakdown  of  the  Overlap  Between  2013  WEOA  and  2009  WEOA 


Experienced  Racial  Discrimination 
(Victims)  in  2009  and  in  2013  Sample 

Did  Not  Experience  Racial  Discrimination 
(non-Victims)  in  2009  and  in  2013  Sample 

Frequency 

Percent 
of  Total 

2013 

Unweighted 

Response 

Rate 

(percent) 

2013 

Weighted 

Response 

Rate 

(percent) 

Frequency 

Percent 
of  Total 

2013 

Unweighted 

Response 

Rate 

(percent) 

2013 

Weighted 

Response 

Rate 

(percent) 

Total 

563 

100 

44 

48 

3,194 

100 

45 

51 

Race/Ethnicity 

Non-Hispanic 

White 

7 

1 

71 

99 

104 

3 

57 

64 

Minority 

556 

99 

43 

43 

3,090 

97 

44 

48 

Black 

88 

16 

38 

36 

281 

9 

52 

52 

Hispanic 

43 

8 

53 

50 

246 

8 

44 

42 

Asian 

112 

20 

46 

48 

443 

14 

51 

52 

All  Other  Races 

313 

56 

42 

44 

2,120 

66 

42 

47 

Service 

Army 

125 

22 

38 

41 

459 

14 

41 

51 

Navy 

139 

25 

42 

56 

667 

21 

41 

54 

Marine  Corps 

79 

14 

43 

44 

588 

18 

47 

46 

Air  Force 

106 

19 

44 

41 

966 

30 

40 

45 

Coast  Guard 

114 

20 

50 

55 

514 

16 

58 

60 

Paygrade 

E1-E4 

215 

38 

36 

38 

1,084 

34 

32 

38 

E5-E9 

234 

42 

47 

55 

1,332 

42 

50 

57 

W1-W5 

9 

2 

33 

22 

74 

2 

61 

78 

01-03 

60 

11 

53 

49 

461 

14 

49 

47 

04-06 

45 

8 

51 

50 

243 

8 

61 

63 

Gender 

Male 

446 

79 

45 

46 

2,592 

81 

45 

54 

Female 

117 

21 

38 

55 

602 

19 

42 

39 

Table  12  shows  the  2013  WEOA  response  rates  (unweighted  and  weighted)  by 
demographic  subgroups  unweighted  response  rates  for  the  2013  survey  by  response  to  the  2009 
survey’s  Racial  Discrimination  question.  The  top  row  shows  that  response  rates  for  prior  victims 
and  non-victims  are  very  similar  (45%  versus  44%  unweighted  and  51%  versus  48% 
weighted)10.  When  we  examine  the  ‘percent  of  total’  columns  for  victims  and  non-victims,  the 
largest  differences  in  composition  are  in  race/ethnicity  and  Service.  Although  the  overall 
minority  proportion  is  similar  (99%  versus  97%),  Black  and  Asian  make  up  a  higher  percentage 


10  It  is  important  to  note  that  the  analysis  is  made  almost  exclusively  on  the  minority  group  (3,646  out  of  3,757  of 
the  resampled  members  are  minorities  due  to  the  intentional  oversampling  of  minorities  for  the  WEOA  surveys) 
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of  victims  (16%  versus  9%  for  Black  and  20%  versus  14%  for  Asian)  while  All  Others  Races 
have  the  opposite  effect  and  make  up  the  largest  percentage  in  both  but  make  up  a  smaller 
percentage  (56%  versus  66%)  in  the  victims  group.  For  Service,  victims  are  disproportionately 
Army  and  Navy.  While  demographic  breakdowns  differ  based  on  prior  reporting  of 
victimization,  NRB  will  only  result  if  the  response  rates  for  these  subgroups  differ  between  those 
who  experienced  Racial  Discrimination  and  those  who  did  not. 

Two  competing  hypotheses  for  WEOA  surveys  may  be  1)  victims  are  more  likely  to 
respond  to  “tell  their  story”  or  make  the  military  aware  of  this  serious  problem,  or  2)  victims 
avoid  this  survey  because  it  may  cause  them  to  re-experience  a  traumatic  event.  Although  it’s 
encouraging  that  response  rates  for  victims  and  non- victims  are  similar,  estimates  of  Racial 
Discrimination  rates  could  still  be  biased  if  these  similarities  are  influenced  by  demographic 
differences  between  subgroups.  However,  if  these  response  propensities  are  explained  by 
demographic  variables,  the  weighting  also  reduces  nonresponse  bias.  For  instance,  some 
demographic  subgroups  that  disproportionately  experience  Racial  Discrimination,  such  as  junior 
enlisted,  are  also  traditionally  poor  respondents.  Therefore,  as  described  above,  the  slightly 
lower  weighted  response  rates  for  victims  (48%  versus  51%)  may  be  a  result  of  their 
demographics  (38%  E1-E4  compared  with  34%  for  non-victims)  rather  than  their  experience. 
Because  DMDC  accounts  for  paygrade  during  weighting,  the  slightly  different  response  rates  by 
victimization  may  be  accounted  for  due  to  the  correlation  between  paygrade  and  experience. 

To  investigate  further,  DMDC  ran  unweighted  and  weighted  logistic  regression  models 
where  the  dependent  variable  was  response  to  the  survey  and  the  independent  variables  were 
Service,  paygrade,  gender,  race/ethnicity,  and  a  dummy  variable  for  prior  Racial  Discrimination 
(victimization).  Table  13  shows  the  weighted  logistic  regression  because  the  weights  account  for 
differences  in  the  composition  of  the  two  groups  (as  mentioned  earlier,  the  higher  proportion  of 
Black  and  Asian  in  the  “experienced”  group),  and  we  therefore  consider  the  weighted  model 
better. 


Table  13  shows  the  output  from  the  weighted  logistic  regression  using  SAS  PROC 
SURVEYLOGISTIC.  The  analysis  of  statistical  significance  and  the  odds  ratios  used  in  Section 
111  can  be  used  here  as  well.  The  results  show  many  typical  conclusions,  such  as  all  paygrade 
groupings  are  more  likely  to  respond  to  the  survey  than  the  E1-E4  reference  group  (all  odds 
ratios  are  greater  than  1).  All  services  are  more  likely  to  respond  to  the  survey  than  Army,  and  in 
particular  the  Coast  Guard  and  Air  Force  (odds  ratios  of  2.907  and  1.883,  respectively).  After 
controlling  for  the  other  independent  variables,  prior  experience  does  not  affect  one’s  likelihood 
to  respond  to  a  later  survey.  The  odds  ratio  is  very  close  to  one  (1.096)11,  and  far  from 
statistically  significant  (p  =  0.6889),  and  we  conclude  that  prior  victimization  has  a  very  small 
influence  on  future  response  to  the  2013  WEOA. 


11  The  odds  ratio  is  0.973  in  the  unweighed  model,  and  also  far  from  statistically  significant. 
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Table  13. 

Logistic  Model  Predicting  Response  to  the  2013  WEOA  Survey  (weighted,  n=3,757) 


Parameter 

Estimate 

Standard 

Error 

Wald 

Test 

Statistic 

df 

P-value 

Odds 

Ratio 

95  Percent  C.I.  for 
Odds  Ratio 

Lower 

C.I. 

Upper 

C.I. 

Racial  Discrimination 

0.1603 

1 

0.6889 

Did  Experience 

0.0457 

0.1141 

0.1603 

1 

0.6889 

1.096 

0.7 

1.714 

Race/Ethnicity 

12.6044 

4 

0.0134 

Black 

0.00407 

0.1996 

0.0004 

1 

0.9837 

0.674 

0.32 

1.418 

Hispanic 

-0.4492 

0.1826 

6.0529 

1 

0.0139 

0.428 

0.212 

0.866 

Asian 

0.228 

0.1369 

2.7733 

1 

0.0958 

0.843 

0.445 

1.598 

All  Other  Races 

-0.1816 

0.1166 

2.4256 

1 

0.1194 

0.56 

0.303 

1.035 

Gender 

3.9997 

1 

0.0455 

Female 

-0.2467 

0.1233 

3.9997 

1 

0.0455 

0.611 

0.377 

0.99 

Paygrade 

18.614 

4 

0.0009 

E5-E9 

0.1591 

0.1943 

0.6708 

1 

0.4128 

2.421 

1.556 

3.765 

W1-W5 

0.7845 

0.522 

2.2585 

1 

0.1329 

4.524 

1.207 

16.958 

01-03 

-0.3808 

0.2641 

2.078 

1 

0.1494 

1.411 

0.738 

2.696 

04-06 

0.1621 

0.3417 

0.2249 

1 

0.6353 

2.428 

1.032 

5.712 

Service 

23.6554 

4 

<.0001 

Navy 

-0.0203 

0.1469 

0.0191 

1 

0.8902 

1.498 

0.918 

2.444 

Marine  Corps 

-0.4071 

0.2408 

2.8579 

1 

0.0909 

1.017 

0.507 

2.04 

Air  Force 

0.2086 

0.1812 

1.3265 

1 

0.2494 

1.883 

1.068 

3.32 

Coast  Guard 

0.643 

0.1513 

18.0494 

1 

<.0001 

2.907 

1.784 

4.739 

Constant 

-0.1766 

0.199 

0.7878 

1 

0.3748 

Summary  of  Analyzing  Whether  Past  Victims’  Respond  to  Later  WEOA  Surveys  at 

Different  Rates 

To  assess  NRB,  DMDC  checked  whether  Racial  Discrimination  victims  may  be  more  (or 
less)  likely  to  respond  than  non-victims  by  tracing  prior  WEOA  survey  respondents  and 
examining  their  response  rates  to  the  2013  WEOA.  DMDC  also  ran  logistic  regression  models 
where  the  key  independent  variable  was  a  dummy  variable  representing  prior  victimization. 

There  were  3,757  2009  WEOA  respondents  that  were  sampled  for  the  2013  WEOA  survey.  Of 
the  3,757  members,  563  had  reported  racial  victimizations  while  3,194  had  not.  Prior  victims 
and  non- victims  had  extremely  similar  response  rates  to  the  2013  WEOA  (44%  versus  45% 
unweighted  and  48%  versus  51%  weighted).  We  caution  against  drawing  conclusions  from  this 
study  alone  due  to  the  small  number  of  prior  victims,  but  the  similar  unweighted  and  weighted 
2013  response  rates  between  the  two  groups  and  the  lack  of  significance  of  prior  victimization  on 
response  rates  from  our  logistic  regression  models  provides  no  evidence  of  NRB  in  the  2013 
WEOA  estimates. 
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Section  VI: 

Analyze  Mean  Armed  Forces  Qualification  Test  Scores  Between 
Active  Duty  Population  and  WEOA  Survey  Respondents 

The  Armed  Forces  Qualification  Test,  or  AFQT,  consists  of  the  following  four  sections 
from  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB):  Word  Knowledge,  Paragraph 
Comprehension,  Arithmetic  Reasoning,  and  Mathematics  Knowledge.  The  scores  from  these 
four  sections  make  up  the  Military  Entrance  Score,  which  is  also  known  as  the  AFQT.  The 
AFQT  score  is  used  to  determine  eligibility  for  entrance  into  the  Armed  Services,  as  well  as  your 
training  potential  with  the  Armed  Services.  DMDC  compared  weighted  estimates  of  AFQT  score 
for  WEOA  respondents  to  the  known  population  value  for  the  corresponding  active  duty 
population.  If  the  weighted  survey  estimates  differed  substantially  from  the  mean  AFQT  score 
in  the  population,  this  would  provide  evidence  of  possible  NRB  in  2013  WEOA  estimates.  Note 
that  DMDC  does  not  currently  use  AFQT  score  as  an  administrative  variable  when  calculating 
survey  weights.  If  weighted  estimates  from  survey  respondents’  mean  AFQT  score  exceeded  the 
active  duty  populations,  this  would  show  that  ‘intelligent’  Service  members  respond  to  surveys  at 
higher  rates.  If  intelligence  were  correlated  with  other  attributes  and  experiences  (e.g.,  racial 
harassment),  then  survey  estimates  may  be  biased  due  to  our  failure  to  include  AFQT  in  our 
weighting  models. 

DMDC  focused  on  the  AFQT  percentile  and  ran  PROC  SURVEYMEANS  on  the 
1,150,283  active  duty  members  in  the  population  as  well  as  the  13,895  respondents  to  the  2013 
WEOA  that  had  an  AFQT  percentile  on  file.  Only  enlisted  members  have  AFQT  scores; 
therefore,  the  analysis  was  only  performed  on  a  subset  of  the  population  and  survey 
respondents.12 

Table  14  shows  the  mean  AFQT  percentile  overall  and  by  subgroups  for  the  entire 
enlisted  population  versus  the  weighted  mean  based  on  the  13,895  2013  WEOA  respondents. 


12  98.9%  of  enlisted  members  in  the  sample  had  an  AFQT  percentile  on  file  with  the  others  having  an  “Unknown” 
value. 
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Table  14. 

Comparison  of  Mean  AFQT  Percentile  ( Active  Duty  Population  versus  Survey  Respondents ) 


Variable 

Population 

Survey  Respondents 

Size 

Mean  AFQT 
Percentile 

Size 

Weighted  Mean 
AFQT  Percentile 

Overall 

1,150,283 

63 

13,895 

64 

Race/Ethnicity 

Non- 

minority/White 

709,014 

67 

1,609 

68 

Minority 

441,269 

57 

12,286 

59 

Black 

198,241 

53 

2,487 

55 

Hispanic 

144,121 

57 

2,384 

59 

Asian 

38,727 

61 

1,980 

62 

All  Other  Races 

60,180 

65 

5,435 

66 

Service 

Army 

435,956 

59 

3,345 

62 

Navy 

254,292 

65 

3,790 

65 

Marine  Corps 

171,602 

62 

2,377 

65 

Air  Force 

256,432 

67 

2,962 

67 

Coast  Guard 

32,001 

68 

1,421 

69 

Paygrade1 

E1-E4 

611,066 

63 

5,871 

66 

E5-E9 

539,217 

62 

8,024 

62 

Gender 

Male 

986,226 

63 

11,201 

65 

Female 

164,057 

60 

2,694 

61 

!Note  that  only  enlisted  members  have  an  AFQT  percentile  on  file. 


Table  14  shows  that  the  weighted  mean  AFQT  percentile  of  the  respondents  is  nearly  the 
same  as  that  of  the  population  (64%  versus  63%).  Although  similar,  the  weighted  mean  from  the 
survey  respondents  is  always  slightly  greater  than  or  equal  to  the  population  mean  across  all 
other  domains  shown  in  the  table.  If  anything,  we  conclude  that  more  ‘intelligent’  members 
respond  to  the  WEOA  survey  at  slightly  higher  rates,  but  again  differences  are  so  small  it  is 
unlikely  that  this  contributes  much  toward  NRB. 

Summary  of  Analyzing  Mean  Armed  Forces  Qualification  Test  Scores  Between 
Active  Duty  Population  and  WEOA  Survey  Respondents 

DMDC  investigated  whether  respondents  to  the  WEOA  had  systematically  different 
AFQT  scores  than  nonrespondents  after  controlling  (through  weighting)  demographic 
differences  between  survey  respondents  and  nonrespondents.  If  the  respondents  systematically 
differ  from  nonrespondents  and  the  differences  could  not  be  controlled  by  survey  weighting, 
estimates  of  any  parameter  correlated  with  intelligence,  as  measured  by  AFQT,  are  likely  biased. 
In  summary,  DMDC  concludes  that  this  study  provides  very  little  evidence  of  NRB  because  the 
weighted  estimates  almost  exactly  match  the  known  population  values. 
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Appendix  A. 

Creation  of  Racial/Ethnic 
Harassment/Discrimination  Rate 


Creation  of  Racial/Ethnic  Harassment/Discrimination  Rate 


For  the  2013  WEOA,  DMDC  created  the  Racial/Ethnic  Harassment/Discrimination  rate 
based  on  one  of  two  criteria  spanned  over  four  separate  questions. 

1.  Harassment:  The  member  must  have  answered  “Once  or  Twice”,  “Sometimes”,  or 
“Often”  on  any  sub  item  a-r  on  Question  28  and  answered  “Some”  or  “All”  on 
Question  29. 

(OR) 

2.  Discrimination:  The  member  must  have  answered  ‘Yes’,  and  my  race/ethnicity  was/is 
a  factor”  on  any  sub  item  a-u  on  Question  31  and  answered  “Some”  or  “All”  on 
Question  32a  indicating  “Racial/ethnic  discrimination.” 

The  questions  involved  in  creating  the  Racial  Discrimination  rate  can  be  seen  in  Figure 
A-l,  Figure  A-2,  Figure  A-3,  and  Figure  A-4. 


Figure  A-l. 
Question  28 


28.  How  frequently  during  the  past  12  months  have  you  been  in  circumstances  where  you  thought 

•  Military  Personnel  (Active  Duty  or  Reserve) 

-  on-  or  off-duty 

-  on-  or  off-installation;  and/or 

•  DoD/DHS  Civilian  Employees  and/or  Contractors 

—  In  your  workplace  or  on  or  off  your  installation/ship...  Mark  one  answer  for  each  item. 


Often 

Sometimes 

Once  or  twice 

Never 

a. 

Made  unwelcome  attempts  to 
draw  you  into  an  offensive 
discussion  of  racial/ethnic 
matters? . 

El 

El 

El 

El 

b. 

Told  stories  or  jokes  which 
were  racist  or  depicted  your 
race/ethnicity  negatively? . 

El 

El 

El 

El 

c. 

Were  condescending  to  you 
because  of  your  race/ 
ethnicity? . 

El 

El 

El 

El 

d. 

Put  up  or  distributed 
materials  (for  example, 
pictures,  leaflets,  symbols, 
graffiti,  music,  stories)  which 
were  racist  or  showed  your 
race/ethnicity  negatively? . 

El 

El 

El 

El 

e. 

Displayed  tattoos  or  wore 
distinctive  clothes  which 
were  racist? . 

El 

El 

El 

El 

f. 

Did  not  include  you  in  social 
activities  because  of  your 
race/ethnicity? . 

El 

El 

El 

El 

g- 

Made  you  feel  uncomfortable 
by  hostile  looks  or  stares 
because  of  your  race/ 
ethnicity? . 

El 

El 

El 

El 

h. 

Made  offensive  remarks 
about  your  appearance  (for 
example,  about  skin  color) 
because  of  your  race/ 
ethnicity? . 

El 

El 

El 

El 

i. 

Made  offensive  remarks 
about  your  accent  or 
language  skills? . 

El 

El 

El 

El 

j- 

Made  remarks  suggesting 
that  people  of  your  race/ 
ethnicity  are  not  suited  for 
the  kind  of  work  you  do? . 

El 

El 

El 

El 

k. 

Made  other  offensive 
remarks  about  your  race/ 
ethnicity  (for  example. 

El 

El 

El 

El 

referred  to  your  race/ 
ethnicity  with  an  offensive 
name)? . 

1.  Vandalized  your  property 
because  of  your  race/ 
ethnicity? . 

E 

E 

E 

E 

m.  Hazed  you  (for  example, 
experienced  forced  behaviors 
that  were  cruel,  abusive, 

oppressive,  or  harmful) 
because  of  your  race/ 
ethnicity? . 

E 

E 

E 

E 

n.  Bullied  vou  (for  example, 
experienced  verbal  or 
physical  behaviors  that  were 

threatening,  humiliating,  or 
intimidating)  because  of  your 
race/ethnicity? . 

El 

E 

E 

E 

o.  Made  you  feel  threatened 

with  retaliation  if  you  did  not 
go  along  with  things  that 
were  racially/ethnically 
offensive  to  you? . 

E 

E 

E 

E 

p.  Physically  threatened  or 
intimidated  you  because  of 
your  race/ethnicity? . 

E 

E 

E 

E 

q.  Assaulted  you  physically 
because  of  your  race/ 
ethnicity? . 

E 

E 

E 

E 

r.  Other  race/ethnicity-related 
experiences? . 

E 

E 

E 

E 

Figure  A-2. 
Question  29 


29. 


[Ask  if  Any  Q28  a  -  r  GT  Never  |  Do  you  consider  ANY  of  the  behaviors  which 
you  marked  as  happening  to  you  in  the  previous  question  to  have  been  racial/ 
ethnic  harassment? 


13 

13 

13 


None 

Some 

All 


Figure  A-3. 

Question  31 

31.  During  the  past  12  months,  did  any  of  the  following  happen  to  you?  If  it  did,  do  you  believe  your  race/ 
ethnicity  was  a  factor?  Mark  one  answer  for  each  statement. 


No,  or  does  not  apply 

Yes,  but  my  race/ethnicity  was/is  NOT  a 

factor 

Yes,  and  my  race/ethnicity  was/is  a 

factor 

a. 

You  were  rated  lower  than  you 
deserved  on  your  last  evaluation . 

13 

3 

3 

b. 

Your  last  evaluation  contained 
unjustified  negative  comments . 

13 

c. 

You  were  held  to  a  higher 
performance  standard  than  others 
in  your  job . 

13 

d. 

You  did  not  get  an  award  or 
decoration  given  to  others  in 
similar  circumstances . 

13 

e. 

Your  current  assignment  has  not 
made  use  of  your  job  skills . 

13 

f. 

You  were  not  able  to  attend  a 
major  school  needed  for  your 
specialty . 

13 

g- 

You  did  not  get  to  go  to  short  (1- 
to  3-day)  courses  that  would 
provide  you  with  needed  skills  for 
your  job . 

13 

h. 

You  received  lower  grades  than 
you  deserved  in  your  training . 

13 

i. 

You  did  not  get  a  job  assignment 
that  you  wanted  because  of  scores 
that  you  got  on  tests . 

13 

j- 

Your  current  assignment  is  not 
good  for  your  career  if  you 
continue  in  the  military . 

3 

k. 

You  did  not  receive  day-to-day, 
short-term  tasks  that  would  help 
you  prepare  for  advancement . 

3 

1. 

You  did  not  have  a  professional 
relationship  with  someone  who 

3 

3 

3 

advised  (mentored)  you  on  career 
development  or  advancement . 

m. 

You  did  not  learn  until  it  was  too 

late  of  opportunities  that  would 
help  your  career . 

E 

E 

E 

n. 

You  were  unable  to  get  straight 
answers  about  your  promotion 
possibilities . 

E 

E 

E 

o. 

You  were  taken  to  nonjudicial 
punishment  or  court  martial  when 
you  should  not  have  been . 

E 

E 

E 

p- 

You  were  punished  at  your  job  for 
something  that  others  did  without 
being  punished . 

IE 

E 

E 

q- 

You  were  excluded  by  your  peers 
from  social  activities . 

E 

E 

E 

r. 

You  got  poorer  military  services 
(for  example,  at  commissaries, 
exchanges,  clubs,  and  rec  centers) 
than  others  did . 

E 

E 

E 

s. 

You  received  poorer  treatment 
than  you  deserved  from  a  military 
health  care  provider . 

E 

E 

E 

t. 

You  were  harassed  by  armed 
forces  police . 

E 

E 

E 

u. 

You  had  other  bothersome 
experiences  at  your  job . 

E 

E 

E 

Figure  A-4. 

Question  32 

32.  [Ask  if  Any  Q31  a  -  u  =  "Yes,  and  my  race/ethnicity  was  a  factor"  OR  "Yes,  but  my 
race/ethnicity  was  NOT  a  factor"]  Do  you  consider  ANY  of  the  behaviors  which 
you  marked  in  the  previous  question  to  have  been...  Mark  one  answer  for  each 
item. 


All 

Some 

None 

a.  Racial/ethnic  discrimination? . 

E 

E 

E 

b.  Sex  discrimination? . 

E 

E 

E 

c.  Religious  discrimination? . 

E 

E 

E 

d.  Other  type  of  discrimination? . 

E 

E 

E 
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